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ABSTRACT 



The course loading forecasts for the Operations Research 
Department published by the Naval Postgraduate School 
Programming Office are investigated for measures of 
predictive accuracy by course and curricula. Both course 
and curricula forecasts are categorized as problem causing, 
reasonably accurate, or undefined due to limited informa- 
tion. The paucity of the data precludes the use of sophis- 
ticated statistical techniques to identify any permanence in 
effects thereby limiting results to indications of 
predictive performance. A management aid is developed 
consisting of a data base and manipulative APL programs for 
the purposes of updating and displaying current information 
while identifying possible synergistic interactions between 
data subcatagories. Appendix A contains the Users Guide for 
this management aid. These results can be applied to the 
current forecast to aid the department chairman in creating 
the faculty teaching schedule. 
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I. ^^GROOND 



A. PEOBIEM HISTORY 

The Naval Postgraduate School uses a complex prediction 
system to forecast expected student demand for courses over 
the upcoming year. This forecast is computed using various 
inputs from several sources during the summer quarter (July 
to September) and covers the following academic year - fall 
through summer quarters inclusive. The tabled output of 
course loading predictions so computed is used by the 
various Department Chairmen to schedule faculty teaching 
responsibilities and research time for that year. As with 
any forecasting system, this set of course loading predici- 
tons is not exact. Various errors are introduced either 
from the simplif icat icn of the process required to get the 
model to a managable size or from erroneous input. The 
Operations Research Department Chairman noticed an inaccu- 
racy problem when a class with a predicted size of approxi- 
mately 30 was scheduled and approximately 75 students showed 
up, causing a disruption of faculty teaching plans. 
Believing that a problem existed, he posted a request to 
research the scope of the inaccuracies as pertaining to the 
courses taught by the Operations Research Department. The 
study's goal would be to determine if a significant problem 
existed and, if so, to provide either a series of adjust- 
ments to be made to the present predictions or a better 
prediction method. 

B. PRESENT SYSTEM 

Until 1972, the scheduling and course prediction system 
was run by hand using "corporate memory"; the historical 
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demand for courses as recalled by the curriculum representa- 
tives involved. These recollections were used to prepare a 
proposed schedule that the students and curricular offices 
used and modified as reguired. By 1972, this system had 
just about reached the limits of its capabilities. At that 
time, the scheduling problem was passed on to a newly hired 
NFS computer program designer^ who tailored a predictive 
computer model to the school’s needs. That model, with 
modifications, is still in use today. 

1 , Sch eduli na ^ P redictio n P roblems 

The problem faced in 1971 is the same today - How 
can we accurately predict three items; 

• The courses reguired to match student demand 

• When those courses should be taught 

• How many students will attend 

Two types of student inputs are available; those who are not 
yet onboard (with or without orders) , and those presently 
attending the school in any given capacity. Both types of 
students are designated as attending a given curriculum, and 
from that designation a standard program of courses is 
assigned. This assignment leads to the course scheduling 
output. Since the two input groups are significantly 

different with respect to treatment within the model itself, 
they will be separately discussed. 

a. Students not yet onboard 

The Office of the Chief of Naval Operations 
publishes a yearly quota plan, [Ref. 1], which directs the 
Commander, Naval Military Personnel Command to provide the 
officer personnel to fill the allocated graduate education 
billets. The fiscal year 1985 quota summary is shown in 

^Mike Troian, Special Assistant for Programs: 
working under Code 0 13: J. Barron, Capt. USN (ret) 



presently 



table I . These quotas are further broken down to show, by 
professional designator (i.e. Surface Warfare, NFO, 
Cryptology - - .) , what curricula the officers will attend. 
Historically, however, these quotas have been guidelines and 
not strictly followed. Additionally, Naval Military 
Personnel Command (NMPC) gives only a rough idea of in which 
quarter an officer will arrive during a given year. This in 
turn drives the input problem experienced at NPS. Through 
liason with NMPC, the school identifies as specifically as 
possible just who has orders to attend NPS either in hand or 
being written. Quota shortfalls tend to occur in the ’’tech- 
nical” curricula due to a lack of available, qualified 
personnel. Conversly, surpluses occur in the "ncn- 
technical” curricula. As with any military organization, 
operational commitments such as maintaining ships in the 
Eastern Mediterranian Sea or Indian Ocean adversly affect 
the arrival of Surface Warfare Officers (who consitute the 
largest group in the NPS student population) . 

The school folds all of this quota information 
and experience into a ’’best guess” of the number of students 
arriving per curriculum per quarter. 2 As this process is 
being completed, the curricular officers designate the paths 
these new arrivals will follow. This is done either by 
service (i.e. USN students take the following courses: . 

.) or by past experience (i.e. 28 % of incoming students need 
refresher math followed by . . .). The resulting linking of 
numbers of students and assigned courses provides one type 
of input to the scheduling model. 



2 Additionally , the programming office contacts indi- 
vidual service and foreign representatives to further refine 
the. expected inputs (since [Bef. 1 ] pertains only to OSN 
officers). These organizations provide less reliable data 
mainly because the fiscal year for other US services starts 
in January. The planning for their personnel transfers is 
therefore not complete until October or November. 
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TABLE I 

Summary table from £Ref. 1] 
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separate plan - published 



h. Students presently attending NPS 

The scheduling process for those students 
already onboard is somewhat simpler. Since they are already 
designated as attending a given curriculum over a known 
period of time, the only area in which significant problems 
are found is that of elective scheduling. Since the 
prediction is completed a full year in advance, out-guarter 
electives may not have been correctly selected by the 
student. His interests or his required course load may 
change, thereby changing any electives he has selected. 
Additionally, the course demand may be such that the course 
is withdrawn from the list of available courses. In all 
cases, however, course selection input covering the upcoming 
year is made by the student through his curricular officer 
to the scheduling office. 

2. Scheduling Predictio n Sys tem Use 

Once elective selections have been made by the 
onboard students and the numbers of new inputs per curric- 
ulum are entered, the course loading prediction model 
applies standard course matrixes to generate the overall 
student course demand. The scheduling system is demand 
based, meaning this derived demand drives the faculty hiring 
and scheduling process. The budget process also is based on 
the course demand. Teaching funds are allocated for all 
instructors teaching course sections with five or more 
students. The individual department chairmen receive a 
listing of student demands by course and quarter from which 
they create a proposed schedule of courses to be offered. 
This schedule is further adjusted through the allocation of 
teaching funds in a process consisting of an approximately 
month long dialog between the department chairmen, division 
deans, and administrative representatives from the 
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programming office. The goal of this process is to create an 
acceptable schedule of courses to be offered balanced 
between annual operating budget funds, student course 
demands and faculty research requirements. Through this 
scheduling process, some courses are dropped and others 
added, creating some student movement among the courses 
offered. The goal of the course loading prediction model is 
to minimize the course changes caused by this scheduling. ^ 



3Mike Troian, the Special Assistant for Progr 
from Ingersol Room 375, has routinely conduct 
reviews of the course loading prediction model, 
this review has shown a 90% accuracy in predic 
courses taught with a less than 1% deviation 
number of courses predicted (his figures). As a 
of faculty required and expected funding costs, t 
reportedly excellent. 



ams working 
ed accuracy 
For FY1983. 
ting actual 
from total 
n indicator 
he model is 
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II. IN ITI AL STE PS ^ ANAilSIS 



Before any fact-hased discussion of ’’better” predictions 
could be started, the problem facing the Department Chairman 
had to be defined. A single historical observation of a 
severe prediction error that caused problems is not signifi- 
cant considering the number of predictions and observations 
made. Historical data had to be checked to determine if the 
problem cf inaccurate predictions actually existed or was 
merely assumed based on a seldom occurring event. Since 
predictions were made by course, the obvious choice for 
comparison was the historical course data available from the 
registrar’s office. Not until these steps were completed 
could a search for a tetter method of prediction be started. 

A. DATA ORGANIZATION 

One of the initial steps of the analysis was an organi- 
zation of all the usable data into a guantifiable format. 
Useful information available consisted of the following 
items; 

•course number 

•quarter taught 

•number of students predicted 

•number of students attending 

The eventual goal of the analysis became that of finding 
such permanencies in the data (based on these items) as to 
provide an accurate prediction adjustment. 

1 • Prediction Data 

Prediction data was available both for individual curricula 
inputs into a given course and for the total expected 
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student demand for those courses. This data was presented 
in two formats: summary data and individual course data. 
The summary data (figure 2. 1) existed for academic years 
1982, 1983 and the present. This data provided a given 
years display of the total expected student loading indexed 
by course and quarter. It included a recommended number of 
"sections taught" and the corresponding faculty hours 
required. As seen in the figure, the predictions are made 
for a given academic year, in this case 84 (September 1983 
to August 1984) . The first column contains the alpha- 
numeric course identifier as listed in [Ref, 2]. The 'STD' 
columns refer to the number of students predicted to attend, 
the 'SEC column refers to the number of sections required 
to accomodate those students (maximum section size is 
limited to 30 attendees), and the 'FAC-HRS' columns refer to 
the number of faculty teaching hours needed to teach those 
sections. 
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Figure 2.1 Example of Summary Forecasts 
(as delivered to Dept. Chaxraan). 



For 1983 and 1984, individual course predictions 
were available (figure 2.2). These course predictions listed 
the expected student loading for a given course during a 
given quarter coded by degree awarded. This degree coding 
is found in the "S/D/T" (specialty/degree/time) column of 
the data. The first two letters of this column apply to the 
curriculum specialty, the next two to the degree (MS refers 
to Master of Science). This coding is easily tr ansferrable 
to curriculum number using Naval Postgraduate School 
instruction 1520.19 (figure 2.3). The numbers that occa- 
sionally appear following those four letters indicate when 
curricular officers have not provided an updated list of 
whom they expect for that course at that time in that 
specialty. The numbers mean that the prediction listed came 
from the "standard files” of the programming office; a 
collection of best guesses of the expected number of atten- 
dees loosely based on historical patterns.^ These individual 
course predictions, however, make no mention of segment 
assignments. 



♦ether columns in the individual course prediction data 
are generally self-explanatory with the exception of these 
minor items: The AV column referred to the number of avia- 
tors (Pilots, NFOs, etc.) that attend and is an unused relic 
of some past program. The letters sometimes preceding the 
section column are now meaningless. The section column code 
itself is assigned by the curricular offices to groups of 
incoming students. 
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Figure 2.2 Example of Individual Course Predictions 



"I 



NAVPGSCOLINST 1520.19 CH-1 
21 October 1983 

CODES DSED FOE SCHEDOLING 
A^DF 
JUNE 1983 



CODE 


30 


SPECIALTY 

DEGREE 

CODE SPECIALTY 


360 




RAMS 
ROMS 
ENHS 
RSMS 
RPPH 
EC MS 


OH/SA ARMY 

OR/SA GENERAL 

OR/SA NAVY 

OR/SA SUPPLY CORPS 

OR/SA PHD IN OR 

GROUP OF ANY OF ABOVE 


CODE 


31 






€ 10 




ACMS 


AEECNAOTICAL ENGINEERING 


611 




AXMS 

ATMS 


AERONAUTICAL ENGINEERING (Avionics) 
AEEC ENGINEERING FOR TEST PILOTS 


CODE 


32 






600 




DC MS 


ELECTRICAL ENGINEERING (Communications) 


590 




EAMS 


ELECTRICAL ENGINEERING 


620 




HCMS 

HMMS 


COMMUNICATION MANAGEMENT (Coast Guard) 
COMMUNICATION MANAGEMENT 


595 




KEMS 


ELECTRONIC WARFARE SYSTEMS 


366 




SOMS 


SPACE SYSTEMS OPERATIONS 


591 




SEMS 


SPACE SYSTEMS ENGINEERING 


CODE 


33 






380 




GMMS 


MATH OPTION OF ADVANCED SCIENCE 


381 




GPHS 


PHYSICS OPTION OF ADVANCED SCIENCE 


530 




HTMS 


WEAPONS SYSTEM ENGINEERING 


531 




WSMS 


WEAPONS SYSTEM SCIENCE 


532 




WNMS 


NUCIEAR PHYSICS (Weapons and Effects) 


535 




UXMS 


UNDERWATER ACOUSTICS 


CODE 


331 






525 




IXMS 


ANTISUBMARINE WARFARE 



Figure 2.3 Partial Listing of Degree to Curriculum Codes 
(see [Ref, 3] or Appendix B for complete listing). 
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2 . 



Historical Data 



Historical attendance data was made available by the 
registrar's office. This data, filed by quarters listed the 
courses taught in alpha-numeric order displaying student 
names, curricula, and grades received (see figure 2.4). The 
numbers recorded reflect the number of students completing 
the course vice the number seated on the first day of class. 
Segements are listed separatly. Use of this historical data 
implies a number of actions. First, the predictions are 
demand based and would probably correlate better with the 
number of students registered for the course than with the 
number completing. A strong positive correlation is assumed 
between number of students registered and number completing 
a given course. Second, grade and name information was 
discarded thereby burying any measure of correlation between 
performance and attrition. Third, segment data was collapsed 
to match individual course prediction data. This obscured 
relationships betw-een predicted and actual number of 
sections taught or faculty hours required. Since the goal of 
the analysis was to adjust the predictions of student demand 
vice to explain the reasons an adjustment is necessary, 
these actions were considered acceptable. 



SThe registrar's system is rather unique in its coding 
of this data. 'School Year 84' does not refer to fiscal 
year 84 (normally 1 October 1983 - 3TJ September 1984). 
Instead, 'School Year 84' refers to the September 84 to 
August 85 quarters. As this data was collected, the coding 
of quarterly data fto be discussed later) was translated 
bacK into standard fiscal year notation. Thus, throughout 
the data base used in the analysis, 83.1 refers to the 
September 82 - December 82 quarter of the 1982 - 1983 school 
year, as expected. 
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OFFICIAL GRADE ROSTER PAGE 1 

NAVAL POSTGRADUATE SCHOOL 
MONTEREY, CALIFORNIA 93943 DATE 13 APR 

COURSE NO. OS3602 SEGEMENT 00 QUARTER 1 AY 83-84 

COHEAT MODELS PROF. SHUDDE, REX H. 



3 LECTURE HCURS 1 LABORATORY HOURS 



1. 


NAME 

xxxxxx. 


xxxx 


X 


RANK 

LT 


CORPS/ 

COUNTRY 

USN 


CURR 

#374 


MARK 

A 


2. 


xxxxxx^ 


xxxx 


X 


LCDR 


USN 


#374 


B 


3. 


XXXXXXg 


xxxx 


X 


LCDR 


DSN 


#525 


B 


4. 


xxxxxx. 


xxxx 


X 


LT 


USN 


#525 


B 


5. 


xxxxxx. 


xxxx 


X 


LCDR 


USN 


#525 


A- 


6. 


xxxxxx. 


xxxx 


X 


LT 


USN 


#525 


A 


7. 


xxxxxx. 


xxxx 


X 


LCDR 


USN 


#374 


A- 


8. 


xxxxxx. 


xxxx 


X 


LT 


USN 


#525 


A 


9. 


xxxxxx. 


xxxx 


X 


LT 


USN 


#525 


B 


10. 


xxxxxx^ 


xxxx 


X 


LT 


USN 


#525 


B 


1 1. 


xxxxxx^ 


xxxx 


X 


LCDR 


USN 


#525 


B 


12. 


xxxxxx^ 


xxxx 


X 


LT 


USN 


#525 


B 



FORM 4 



THIS IS THE OFFICIAL GRADE. ROSTER TO BE FILED IN THE 
ACAEEMIC DEPARTMENT 



Figure 2.4 Sample Attendance Data (names removed) 

3 . Coding S yste m 

Before entering the data into computer files, a 
coding system was devised to distill the raw data into its 
usable parts. Excess information such as student names and 
grades could then be discarded. The data consisted of 
predictions and actual attendance figures coupled with iden- 
tifiers. Identifiers required include course number and 
quarter taught. Available data included both total and indi- 
vidual curricula figures. For simplicity of use, a basic 
file was created using the following five-column coding 
scheme: 
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Quarter Curricula# Course# #attended #predicted 

Once these items had been tabulated, an easy subtraction 
(attending minus predicted) provided an error term. By 
ordering (by magnititude of error) and displaying the 
resulting array, a history of many poor predictions was 
observed. This observation confirmed that a problem existed 
and showed a need for further analysis. 

B. NATOEAl DIVISIONS IN CODED DATA 

The large volume of coded data created was divisible 
into several sub-categories. The Operations Research 
Department teaches two types of courses; OA - taken by oper- 
ations analysis major students; and OS - taken by students 
from ether curricula. Initially both of these types were 
taken under consideration for analysis. Later, the OA 
designated courses were dropped from consideration for two 
reasons; the problem of not knowing how many students were 
going to show up was generally solvable within the depart- 
ment; additionally, time constraints limited the scope of 
the areas to be covered. * The OS course data was broken into 
two prediction areas: summary predictions and curricula 
specific predictions. The summary predictions were used to 
show patterns by course; the curricula predictions to show 
patterns by individual curricular offices. 

1 . Summar y Dat a 

When dealing with summary prediction data error 
terms, the goal of the analysis was to provide a functional 
relationship between predictions and attendance by course. 



®04 data collected during the initial stages of problem 
investigation is available. The format is slightly 
different from that of the OS course data because the 

f roblem found for OA dealt with elective selection. The 
actor of interest was service affiliation. For further 
details, see appendix A. 
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Using these derived relationships (hopefully simple addi- 
tions or subtractions ) , the department chairman will be able 
to take the forecast from the programming office (figure 
2.1) and change the figures shown to reflect more accurately 
the expected student load. These more accurate figures can 
then be used to schedule faculty teaching assignments, 
request necessary funding, and accomplish other tasks 
requiring the number of expected students per course. 

2- Curricula Spe cific Data 

Uhen dealing with error terms for individual 
curricula, the goal of the analysis was to provide a func- 
tional relationship between predictions and attendance by 
curriculum. Using these derived relationships, the depart- 
ment chairman will be able to take the forecast for an indi- 
vidual course (figure 2.2) and change the figures shewn to 
reflect more accurately the expected student load by either 
applying the relationship or calling the curricular office 
involved to get better numbers. These historical observa- 
tions will enable tie department chairman both to identify 
curricula that are inadvertently causing scheduling problems 
and to show improvements or deterioration in the scheduling 
process itself. This information can be used both by the 
programming office and the curricular office involved as 
well as the OR Department. 
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III. DE TAI LED ANALY SIS OF OS PM2ICTIVE DATA 



At the completion of the initial steps toward analysis 
previously discussed, a coded series of data arrays were 
available for use. An example of array formatting is shown 
in figure 3.1 The figure of interest was the "error" term 
found in column six. These error terms, representing the 
difference between the actual number of students attending a 
course vs the number predicted, were displayed in various 
groups ordered by magnitude to show the accuracy patterns of 
the forecast. Areas that were explored for effects included 
the curriculum that provided the input, the courses them- 
selves, and the guarter for which the courses were 
predicted. Underlying patterns such as an individuals 
"path" through various OS courses required for his curric- 
ulum were also investigated. The paucity of the data 
precluded the use of sophisticated statistical techniques 
thereby limiting results to indications of areas of concern 
rather than definitive statistical statements. 

A. TBE DATA 

Six element data arrays (as shown in figure 3. 1) were 
created for each years data. These APL variables were 
combined to form a single compilation of all the available 
data (see table II) . The error terms of this compilation 
were used for the analysis. These error terms are discrete 
integer data the size of which are limited by the maximum of 
the actual or predicted columns terms. Distributional 
fitting was not done for two reasons; the population should 
have high internal correlation due to the interrelation 
between course errors caused by student course matrices, and 



26 



CTR 


CUERIC 


CSE 


ACT 


PEE 


(ACT-PE 






** * 




*** 


******* 


8 4. 3 


0 


2103 


0 


1 


-1 


84.3 


535 


2103 


0 


1 


-1 


84.3 


0 


300 1 


25 


0 


25 


84. 3 


368 


3001 


25 


0 


25 


84.3 


0 


3002 


15 


18 


-3 


84. 3 


683 


3002 


0 


1 


-1 


84.3 

• 


825 


3002 


15 


17 


-2 


• 

83. 1 


0 


3602 


7 


7 


0 


83. 1 


374 


3602 


1 


1 


0 


83. 1 


525 


3602 


6 


6 


0 


83. 1 


0 


3603 


20 


21 


-1 


83.1 


595 


3603 


20 


21 


-1 


83. 1 


0 


3604 


18 


16 


2 


83. 1 


365 


3604 


1 


0 


1 



QTE: Quarter taught. 83. 1 refers to the first quarter 
in the Sept. 82 to Aug. 83 school year. 

Currie: Curricula - 0 implys summary data for a given 
course. 

Cse: Course number (all OS courses) . 

Act; Actual number of students listed on final grade 
records as completing course. 

Pre; Predicted number of students (from forecast) . 

Act-Pre; Prediction error term. 



Figure 3.1 Example of Array Format (with definitions) 

API variable used ; OS. 

the sample sizes of those homogeneous populations distilled 
from the data (such as "all 365 students taking OS 3604 in 
the second quarter") were prohibitively small. Two types of 
errors were discovered, predictive errors and accuracy 
errors. A predictive error was defined as either the fore- 
cast predicting no students and students attending or vice 
versa. Embedded in this error type are those courses which 
were not even listed but taught and those courses dropped 
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TABLE II 



APL Variables used in Analysis 
(see Appendix A for details or use) 



VARIABLE 
♦ * 


SHAPE 


DESCRIPTION 


OS82 


69x6 


AY (Academic year) 81-82 data; 
includes summary information only 
(no data for individual curricula) 


0S83 


255x6 


AY 82-83 data; complete listing 


0STEK83 


140x8 


AY 82-83 non-entry course data * 


OS84 


228x6 


AY 83-84 data; only gtrs 1,2 and 3 


OS 


552x6 


Combination of all of above 
(without repetition) 


* see ’Course/Curriculum Interactions and Tests* 
section for complete description. 



from the schedule in the budgeting process. The same divi- 
sion was found by individual curricula (for example, curric- 
ulum 460 (engineering science - preparatory) is never 
predicted in any individual course forecast) . Predictive 
errors exist when columns five or six of the data array 
contain a zero. Accuracy errors are other differences 
between the actual and predicted number of students. 

E. CCOBSE/COREICOLOB INTERACTIONS AND TESTS 

Each individual student at the Naval Postgraduate School 
follows an assigned course matrix. This assignment varies 
by both the program assigned and individual student needs. 
However, all matrices list a series of courses that are 
either required or elecive. The Operations Analysis curric- 
ulum, through its OS course offerings, provides a service to 
other programs by providing basic and advanced pobability 



28 



and statistics training not available in other curricula. 
These service courses are generally required by ether 
curricula as a prerequisite to both further training and 
degree completion. As such, they are taught in a certain 
sequence. The amount of training required differs with 
different programs, so various OS "tracks" are required. 
Sequencing of courses in this manner will cause dependence 
between the error terms of certain courses. To determine 
the dependancy relationship within the data, the course 
matrices for all curricula offered at the school were 
collected and the OS course sequences extracted. These 
sequences were combined to form a series of condensed course 
matrices as applicable to the Operations. Research Department 
(see figure 3.2 - Note; x*s refer to students onboard but 
not taking OS or OA designated courses) . From these 
matrices, patterns emerged. As with any program, students 
started with basic courses and moved to more advanced 
topics. Five basic courses were defined as "entry level" 
courses. One of these courses was required before any 
advanced work was done. Since the entry level courses were 
generally taken early in a student’s time in his pregram, 
and since the entry course was the first time he appeared 
eligible for "prediction", the entry level courses were 
expected to have the highest variance in error terms. This 
proved to be true. As the students spend more time in their 
programs, attrition and schedule changing should level off. 
Additionally, since most students spend more than a single 
year going through their programs, "down-line" predictions 
will be improved merely by the students moving from the "not 
yet onboard" to the more stable "attending NFS" prediciton 
group . 
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Figure 3.2 Condensed Course Matrix Examples 



Definition of entry level courses led to the definition 
of coarse sequence "tracks". A student from a given 
curricula will take an "entry" course - the first time he is 
seen ty the OA Department - and will then take follow-on 
courses tased on his course matrix. Each curriculum has its 



ENTRY 
CCUESZ 
^ ^ ♦♦ 


CURRICULUM 


1 

1 

TRACK (Follow-on) 1 

COURSES- ALL OS j 


OS2103 


365 


3604,3404,3008, 

3603,4602 




366 


3604,3404,3008, 

3603,4602 




374 


3601,3602,3603, 

3604 




525 


3303,3604,3601, 
3401 ,4601,3602 




535 


none 




595 


3604,3003,3603, 

3403,4601 


OS3101 


367 


3004 




620 


3005 




686 


none 




687 


none 




825 


3404,3002 



Figure 3. 3 Example of Entry vs Track 
Course Organization. 

own "track" (See figure 3.3). Using these tracks, it should 
have been possible to derive deterministically the exact 
course loading for those follow on or track courses given 
the course matrixes and attendance figures for the entry 
level courses. Using the FY83 data, this was done. The 
resulting matrix was set up in the following manner: 
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Quarter Currie Cse Act Pre Dpre (Act-Pre) (Act-Dpre) 

Act : Actual number attending 
Pre : Standard predictions 
Dpre : Derived predictions 

A display of the results showed no observable difference 
between the regular error term and the derived error term. 
A Wilcoxian signed rank tesf^ was done to test the 
hypothesis that the error magnitude of the adjusted 
predictions tends tc be smaller than the magnitude of the 
error frem the forecast (see table III) . 



TABLE III 

Wilcoxian Signed Bank Test Results 



Sensitive assumption: Differences independent. 

Assumed large ameunts of varied data negated 
track effects resulting in relatively independent 
difference terms. 

CASE A CASE B 

Variables Tested: X =Curric ACT-DPRE Sum ACT-DPRE 

Y =Curric ACT-PRE Sum ACT-PRE 

H : The X values do not tend to be smaller than 

0 the Y values. 

H : The X values tend to be smaller than 

1 the Y values. 

Decision Rule : Reject at the 5% confidence 

level if TS > 1. 65 

CASE A CASE B 
Test Statistic : .453 -1.455 

Conclusion : Cannot reject null hypothesis. 



■^Taken from [Ref. 5 p. 280 ] 



The results of the test indicated that the adjusted 
predictions were no improvement over the forecast. That 
conclusion indicates that interactions more complex than the 
following of course matrixes are involved in actual student 
movement from course to course. The adjusted prediction 
method was dropped from the analysis as a means of better 
predicting course loading. 

C. CTEEB INTERACTIOHS AND TESTS 

Interactions exist in areas separate from the dependen- 
cies created within a class sequence by the course matrix 
structure. Three main effects were studied. First, 
curricula may have individual patterns across time. Some 
groups may consistently have an optimistic input that causes 
an overestimation of number of students wherever those 
groups appear. Second, individual courses may show patterns 
of over- or underestimation. Lastly, a quarterly effect on 
errors may appear due strictly to the elapsed time between 
the prediction and the actual class. To provide an initial 
look at the data to show such patterns, two programs were 
written.® These programs were designed to sort the data into 
the catagories of interest, specifically curriculum, course, 
and quarter. The resulting groupings showed that there were 
patterns in each catagory, but that the number of observa- 
tions available to base conclusions upon made those patterns 
statistically insignificant. Through clever data selection, 
a series of observations was gleaned to test for curriculum 
and course effects using median polish technique as detailed 
in [Bef. 4]. The results of that test were inconclusive 



®These programs and others used in the analysis are 
fully explained and documented in appendix A. That appendix 
also contains instructions for the maintenance and use of 
the data base created, and procedures for gaining access to 
same. 
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The test for 



showing no temporal stability (see table IV) 
quarterly effects was not done due to the 
data . 9 



^Complete data existed only for FY83. 
sample was evaluated as too unrepresentative 
cally significant concerning quarterly 
various years. 



paucity of the 



This limited 
to be statisti- 
effects across 
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The iredian polish technique works well when dealing with the 
•correct' data (ie. a full, singular entry rectangular matrix). 

In the rare case where this type of data existed, results greatly 
differred in cases that should be similar in effect. 



IV. CONCIOSIONS 



The problem as defined was to aid the Department 
Chairman in his task of scheduling and budgeting for faculty 
teaching responsibilities by providing either a better 
prediction method or adjustments to the present forecasts. 
Irom the scheduler's standpoint, jumps across any multiple 
of thirty students cause new sections to be created or 
dropped, and courses with a demand of less than five 
students are dropped. To the scheduler or budget office, 
these are significant numbers. The Department Chairman, 
however, cares little about the actual number but much about 
the accuracy of the prediction. Since predictions are made 
by course and broken down within individual courses by 
curriculum, adjustments were found for courses and 
curricula. Prediction data were the limiting quantities; 
predictions by curricula exist only as far back as FY83. 
This limited the scope of the investigation to 1 and 3/4 
years. ^?ith this limited history, indications of problem or 
non-problem areas are the only adjustment/accuracy measures 
possible. 

A. CCHClOSIONS BY COOESE 

^ ♦ Pro gram Output Dtiliza t io n a nd D isplays 

Indications of predictive accuracy by course were 
obtained using the 'CCURSEP' and 'COURSE' programs detailed 
in Appendix A. The 'COORSEP' (standing for course patterns) 
program was used first. This program takes the input 
matrix, in this case the variable 'OS' containing all of the 
data so far available, and displays it grouped by course in 
decreasing order of predictive error magnitude (see figure 
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Figure 4.1 Sample Output from COURSEP Program 

4.1 for a single course example). lo This output is quickly 
scanned for * interesting' courses - those with either 
significant problems or very small, infrequent errors. 
Those courses with problems are further checked to determine 
if there is any pattern to the errors observed. Many 
negative numbers will indicate an overestimation problem 
(prediction greater than actual attending) , conversely posi- 
tive numbers indicate underestimation. Quarters may be 
split indicating large errors for some, small for others. 
Curricula may be grouped together indicating prediction 
performance differences. In the example shown (Figure 4. 1) , 
error magnitude is small and signs vary, indicating rela- 
tively minor deviations from the predicted values. Had 
these errors been large, the following observations could 



i°In this display (and all others), the quarter is shown 
as a negative number. This is done strictly as a formatting 
aid. In API, a space is added to each column in the output 
display to allow negative signs. The error terms occasion- 
ally had no negatives. When this occurred, the output 
display deleted the six spaces 'saved' for the sign. This 
caused an irritating misalignment in the columns of the 
display. The easy correction was to ensure a negative 
existed in all cases, thus the quarter coding. 
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have teen made: 

• Third quarter errors tend to be higher than 

first quarter errors. 

• No curricula appears to do any better or worse 

than any other, though 374 may tend to over- 
estimate in their predictions. 

To further detail predictive behavior within a 
course, the 'COURSE' program is utilized. This program 
takes the same matrix previously investigated (in this case 
'OS') and also displays it grouped by course, but with 
various catagorical presentations made. Output may be 
displayed in quarterly or curricula subcategories (through 
answering program prompts) , With the curricula option 
selected, the output compares patterns of individual 
curricula (see figure 4.2). Statistics displayed include 
the mean and variance of the error terms for individuals 
curricula, and predictive, accuracy and no error percentages 
also by curricula. To avoid counting data twice, summary 
data entries (column 2=0) are disregarded. From scan of 
the example (figure 4.2) , the following conclusions can be 
drawn : 

• Curriculum 525 provides the majority of the 

input for OS3602, and their predictions 
are excellent indicating no problem with 
the course, 

• Curriculum 374 may show a pattern of over- 

estimation . 

• Curriculum 686 (with 100% prediction errors) 

normally will not provide inputs to this 
course. 
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Figure 4.2 Sample Output from COOESE Program 
Options : Curricula Grouping with Error Percentages. 

The curricula option of the ’COURSE’ program appears 
to be the most valuable display available. The information 
shown is directly transf errable to the prediction data as 
presented (refer back to figures 2.1 and 2.2). 

With the quarter option selected, the output 
combines all of the data into four catagories depending on 
the quarter in which the course has been taught (figure 
4.3) . The same statistics are displayed, this time for each 
quarter. Again, summary data entries are disregarded to 
avoid duplication of data. From a scan of this figure, the 
following conclusions can be drawn: 
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Figure 4.3 Sample Output from CODBSE Program 
Options : Quarter Grouping with Error Percentages. 

• OS3602 has onlj been taught during the fall 

and spring quarters. 

• Fall quarter predictions tend to be exact. 

• Spring quarter predictions are worse, but not 

to such an extent as to cause problems. 



2* * D ispla y Based C onclusion s by Cours e 

Dsing the previously discussed displays, the courses 
were separated into three catagories of interest: those 
with indications of significant prediction accuracy 



40 



and 



problems, those with relatively minor problems if any, 
those with too little information to draw conclusions (see 



TABLE V 

Course Results 



Problem Courses 



2101 

3001 

3102 

3105 



2102 

3008 

3103 

3404 



2103 
3 101 
3 104 
3702 



Inadequate Data 



•Good’ Courses 



3002 

3303 

3602 

4601 



3003 
340 1 



3005 

3403 



3302 

4602 



3004 

3402 

3603 

4701 



3006 

3601 

3604 



table V) . Predictions made for courses can be checked 
against this listing for an indication of their expected 
performance. Should more detailed information be needed. 
Appendix C contains the results based on summary data from 
81-82, complete 82-83 data, and fall, winter and spring 
quarters from the 83-84 school year. 

E. COBCIDSIONS BY CDBRICULA 

• 1 • Out put Dtilization and Displays 

Indications of predictive accuracy by curricula were 
obtained using the 'CDRRICP' and *CDRRIC* programs detailed 
in Appendix A. The 'CDRRICP* (standing for curricula 
patterns) program was used first. This program takes the 
input matrix, 'OS' as before, and displays it grouped by 
curricula in decreasing order of predictive error magnitude 
(see figure 4.4 for a single curriculum example). This 
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Figure 4.4 Saaple Output from CORRICP Program 

output is quickly scanned for 'interesting* courses - those 
with -significant problems or small, infrequent errors. 
Those curricula with significant errors are further checked 
for discernable patterns such as regularly occuring over- or 
underestimation, or quarterly trends. In the example shown 
(figure 4.4), problems appear due to consistent overestima- 
tion. However, given the magnitude of the errors, these may 
be viewed as insignificant. It also appears that fall and 
winter quarters tend to have fewer or smaller prediction 
errors. 

To more closely investigate patterns shown, the 
'CtIRRIC* program is utilized. This program takes the same 
input matrix and displays it further grouped by quarterly or 
course catagories. With the quarter option selected, the 
output compares predictions across time (see figure 4.5). 
From the example shown, the following observations can be 
made; 



• Overestimation errors do increase across 



consecutive quarters. 

• Lower level courses tend to have larger mag 

nitude errors than more advanced courses. 

• OS2103 and OS3401 might not be taken by cur 

riculum 374 students fall quarter (more 
data required) 
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Figure 4.5 Sample Output from CORRIC Pro 
Options : Quarter Grouping with Error Percen 



ram 

ages. 
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with the course option selected, information can be 
gleaned on curriculum 374 's prediction performance for any 
given course (see figure 4.6). Again from the example 
shown, conclusions can be drawn: 

• OS2101 and OS3401 are generally not taken by 

students from curriculum 374 

• Larger errors occur in OS2103 and OS3604 (as 

expected from the course matrix - refer back 
to figure 3.2) 

2 . Cisplav Based Conclusions by Curriculum 

Using the previously discussed displays, the 
curricula were separated into three catagories of interest: 
those with indications of significant prediction accuracy 
problems, those with relatively minor problems if any, and 
those with too little information to draw conclusions (see 
table VI). Predicticns made for courses can be checked 
against this listing for an indication of their expected 
performance. Should more detailed informtion be needed. 
Appendix C contains the results based on summary data from 
81-82, complete 82-83 data, and fall, winter and spring 
quarters from the 83-84 school year. 
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Figure 4-6 Saaple Output from COERIC Program 
Options ; Course Grouping with Error Percentages. 
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TABLE YI 

Curricula Results 



Pr ctlem 

360 368 
531 590 
827 837 



Curricula 



•Good’ Curricula 



374 460 530 
817 819 825 
847 857 



365 366 367 377 525 
595 600 620 813 814 
815 999 



Inadequate data 

372 373 380 440 441 
532 535 570 591 610 
611 683 686 687 
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APPENDIX A 

PROGEAHS AND DATA BASE OSERS MANUAL 

This appendix contains the detailed information reguired 
to effectively run and maintain the programs and data base 
used for historical validation of the Naval Postgraduate 
School course loading forecasts. The purpose of this series 
of programs is to conveniently display historical course 
attendance data versus predicted attendance data. These 
displays can then be translated into simple adjustments to 
be made to the forecast, providing a more accurate figure to 
be used in the faculty teaching assignment process. 

A. ACCESS PROCEDURES 

The data and manipulative programs associated are held 
on the IBM 3033 mainframe at the Naval Postgraduate School 
under user number 1502P. The are contained in an APL work- 
space called ’OSDATA’. Assuming you have a valid user 
account on the main NFS computer system, this workspace can 
be accessed in three steps; linking to user 1502P, accessing 
his disk, and copying the workspace onto your own disk. 
Once you are ’logged on’ to the system, the command 

CP link 1502p 195 195 rr 

will cause a prompt to appear asking for the ’ read’ pass- 
word. Enter ’ps3000’. That password allows access to user 
1502P’s disk. To get the workspace transferred to your cwn 
disk, enter 

acc 195 b/a followed by 

copy osdata vsaplws b = = a 
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B. BDK FBEPARATION INSTEOCTIONS 



To effectively run the programs available, data must exist 
in a six column numerical matrix format coded as shown: 



Column Sample 

Number Ent rie s Comments 

1 83.1 This column corresponds to the quarter 

84.4 from which the data is gleaned. '83.1' 
corresponds to the Fall quarter of the 
1982-1983 school year (as in standard 
fiscal year notation). Similarly, 

84.4 corresponds to the summer quarter 
of the 1983-1984 school year. 



2 



3 



4 



0 This column holds the curriculum code. 

374 '0' means that the entries to follow 

are total number of students predicted 
and total actually enrolled in the 
listed course as compared to '374' 
which means that the figures to follow 
apply only to curriculum 374. 

2103 This column holds the OS course number 
3602 

29 This column holds the actual number of 
0 students attending the course as de- 
scribed by the previous columns. If 
column two is zero, this number is the 
total number of students that attended 
and should correspond to the total 
listed on the grade record sheet held 
by the registrar. If this number 
itself is zero, the course was not 
taught during the quarter shown in 
column one. 
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5 



0 

15 



0 

5 



This column holds the predicted number 
of students that should attend the 
course described. If column two is 
zero, this number is the total number 
of students that were predicted for 
the course and should correspond to 
the totals listed on both the summary 
and individual course forecasts. If 
this number itself is zero, either the 
course was predicted to be empty or 
the course was not predicted. 

This column holds the difference 
between the actual number of students 
attending the course and the number 
predicted (actual minus predicted). 
Negative numbers imply overestimates, 
positive numbers imply underestimates. 



Prior to any program runs, the goal of your analysis 
must be defined. The programs are generally divided into 
two areas, course centered and curricula centered- In the 
course centered area, the 'COOESEP’ program allows a general 
viewing of the error data for chosen courses. This pregram 
displays the data separated into courses ordered by 
decreasing magnitude of the error term (actual minus 
predicted; column six) and is used to check quickly the 
prediction accuracy by course. Then to check for underlying 
patterns by quarter taught or by individual curricula, the 
’COURSE' program can be used. Similarly, in the curricula 
centered area, the 'CURRICP' program allows a general 
viewing of the error data separated into individual 
curricula, and the 'CUEEIC program can be used to further 
distill the information available. 
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In summary, two steps must be taken prior to starting 
program runs: 

• Ensure the data exists in the proper format 

• Decide what information is needed (define goals) 

C. SAMIIE PEOGBAM ED«S 

To demonstrate effectively the varied uses of the 
programs available, a hypothetical problem will be created 
and fully explored. Each program will be discussed in 

detail as it appears in the problem solution, and both the 
inputs reguired and output generated will be shown. 

^ • Proble m State men t 

Curriculum 525, Antisubmarine Warfare, and curric- 
ulum 374, Air-Ocean Tactical Environmental Support, are 
going to combine parts of their programs to create a new 
curriculum: Curriculum 449,5 - Antiwhale Warfare, The 

condensed course matrix for curriculum 449,5 has OS2103, 
CS3604, OS3601, and OS3602 in quarters 2, 3, 4 and 5 respec- 
tively. As the Operations Analysis Department Chairman, you 
know that OS2103 and OS3604 can handle the additional 
students with no problems due to the present size and 
frequency of those course offerings. Your question, with 
scheduling coming up for the next fiscal year, is whether 
the additional 12 students you expect per 'class* will be 
enough to require the offering of an additional section for 
OS3601 or OS3602. The figure of twelve students comes from 
the Air-Ccean curricular office, since curriculum 449.5 will 
be taught as an adjunct of curriculum 374, These students 
will be entering in the Fall and spring quarters. 
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2- Definition of Goals 



The first step in the problem solution is to define 
the information required to answer the question. In this 
case, we need to knew if an additional section is required 
given an influx of twelve additional students. This problem 
defines three questions: 

.• How many students usually take the course in 
question 

• How accurate are those figures 

• How good is the estimate of twelve additional 
students 

Since the first items to be considered involve 
course data, the course related programs will be used. To 
check later the accuracy of the figure twelve, a curricular 
comparison can be made with curriculum 374 (since that 
office is the administrator of the program) using the 
curricula related programs. Additionally, the data used 
must be the most recent and complete information available. 

3. Cou rse Rela t ed Work 

The first step we will take is to get a look at the 
data of interest. The variable ’OS* contains the informa- 
tion we want. The ’CODRSEP’ program most easily shows this 
data. Through proper inputs, we can call up the historical 
performance of OS3601 and OS3602. The 'COURSEP* program 
provides all the prompts necessary (see figure A.1). The 
program output is shown in figure A. 2 . See figure A. 3 for 
a program listing. 

As we can see from figure A. 2, the actual number of 
students that attend each course appears to be somewhere 
around 15. The predictions are presently perfectly adequate 
since they show a requirement for only one section in all 
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cases, which holds true. With the addition of twelve 
students, however, the number of students will be close to 
30, necessitating another section. To more adequately show 
the data, we can look at the ’COURSE’ program sorted by 
curricula. This will break out the summary predictions to 
better show how well the course is predicted. Figure A. 4 
shows the inputs, figure A. 5 the outputs, and figure A. 6 the 
program itself. 

Using only the summary data shown in figure A. 5, we 
can confirm that the actual number of students that attend 
each course is about 15, and the present one section 
requirement holds true. We can further observe a possible 
pattern by quarter (errors appear to increase for the fourth 
quarter predictions as compared to predictions made for 
earlier quarters) . To more adequately show this pattern in 
the data, we can look at the ’COURSE’ program sorted by 
quarter. This will break out the predictions to better show 
quarterly patterns. Figure A. 7 shows the inputs, figure A. 8 
the outputs. 

In summary, through use of the course related 
programs we have found that a course size of approximately 
fifteen is usual for these courses. Predictions, especially 
those made for the fourth quarter, tend to overestimate the 
number of students expected. If the figure of twelve addi- 
tional students is in fact correct, a new section will prob- 
ably not be required even though predicted attendance values 
are in the twenties without the addition of the new 
curr iculum. 
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INPUT PROMPT 



COURSEP OS 

♦ 

0 

♦ 

3601 3602 



* see COURSEP listing for details 
of the prompts 



Figure A.1 Input Syntax : COURSE? 
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r 



OURSE : 
" 83.4 


3601 

0 


3601 


14 


23 


“9 


“ 83.4 


365 


3601 


3 


11 


“8 


“ 82.4 


0 


3601 


14 


21 


“7 


“ 82.2 


0 


3601 


14 


8 


6 


“ 83.4 


374 


3601 




5 


“3 


“ 83.2 


0 


3601 


14 


16 




“ 83.4 


525 


3601 


9 


7 


n 


“ 84.2 


365 


3601 


n 


0 


n 


“ 84.2 


525 


3601 


20 


nn 

A- ^ 


“2 


“ 83.2 


374 


3601 


4 


5 


“1 


“ 83.2 


525 


3601 


10 


11 


“1 


“ 84.2 


0 


3601 


25 


26 


“1 


“ 84.2 


374 


3601 


3 


4 


“1 



OURSE : 
“ 82.1 


3602 

0 


3602 


7 


11 


"4 


“ 83.3 


374 


3602 


3 


5 


"2 


“ 84.3 


374 


3602 


2 


4 


— 2 


“ 84.3 


525 


3602 


12 


10 


n 


“ 82.3 


0 


3602 


15 


14 


1 


“ 83.3 


0 


3602 


15 


16 


“1 


“ 83.3 


686 


3602 


1 


0 


1 


“ 83.1 


0 


3602 


7 


7 


0 


“ 83.1 


374 


3602 


1 


1 


0 


“ 83.1 


525 


3602 


6 


6 


0 


“ 83.3 


525 


3602 


11 


11 


0 


“ 84 . 1 


0 


3602 


12 


12 


0 


“ 84 . 1 


374 


3602 


3 


3 


0 


“ 84.1 


525 


3602 


9 


9 


0 


“ 84.3 


0 


3602 


14 


14 


0 



Figure A. 2 Output froa Input Sequence shown in 
Figure A. 1 : CODRSEp Program. 
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^COURSER 



^ COURSER MATRIX 5 1 ;y ; z 

ci:i ft 

L22 ft 
C3D ft 
ZAl ft 
C5D A 
C6: A 

C7: A 

C8: A 

C9D ft 
CIOD ft 

cii: A 

Z121 ft 
C13: A 
C14D ft 
C15D A 
C161 A 
Z171 A 
C18D A 
C19D ft 
C20] ft 

C213 MATRIX^-MATRIXX ( f matrix ) f “111111 

C223 t'ip' want all courses disrlayed, emter i' 

C23D t'OTHERWISE, ENTER Q TO SELECT SOME SUBSET OF THE COURSES* 

C24D -♦-iUMRixiO 

C253 r ' the following courses are available for selection ;• 

C26D tX<-J<C-kJ<<-UNiauE matrixcjsDD 

C27D t' enter the course ?iumbers you want, enter x to get them all,' 
C28D t'C^-^OTE ; you must choose more than one course)' 

C293 T4-fX4-a 
C30D -»JUMR2 

C313 JUMRl ♦ Y4-f )<<-XCiX«-UNiaUE MATRIXCJ333 
C32D JUMR2:i4-l 
C33D CRSLOORtt' ' 

C34] -»0X\1>Y 

C35D (t 'COURSE ♦ • ) 

Z263 z<-(XCi: = x«trixc ;3D ) / CID matrix 
C37D tZ order 6 
C383 t' ' 

C39D i«-i + l 
C40: r' ' 

C413 4CRSLOOR 
V 



THIS FUNCTION, GIVEN AN INRUT MATRIX IN THE FOLLOWING FORM • 

OTR CURRIC CSE ACTUAL RREDICTED ( AC T U A L - R R E D I C T E D ) 

WILL SORT AMD DISRLAY THE DATA BY CURRICULUM IN THE 
FOLLOWING MANNER * 

COURSE 

DATA ORDERED BY MAGNITUDE OF ERROR TERM 

(ALL QUARTERS WILL BE DISRLAYED AS NEGATIVE NUMBERS, THIS ALLOWS 

FOR CORRECT RLACEMENT OF THE OUTRUT COLUMNS WITHOUT RESORTING 
TO COMRLEX CHARACTER ARRAY M AN I R U L A T I ON S , ) 

THE RROGRAM GOAL IS TO DISRLAY OBVIOUS RATTERNS THAT MAY THEN BE 
LOOKED AT MORE CLOSELY USING THE 'CURRIC' OR 'COURSE' FUNCTIONS, 
COURSES MAY BE SELECTED INDIVIDUALLY IF SO DESIRED, 



Figure A. 3 



♦COORSEP* Program 
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EROHFT 



INEUT 
CCDESZ OS 

* 

0 

* 

3601 3602 

♦ 

0 

♦ 

n 



* see COURSE listing for details 
of the prompts 



Figure A. 4 



Input Syntax ; COURSE 



COURSE : 


3601 










" 83.4 


0 


3601 


14 


23 


“9 


' 82.4 


0 


3601 


14 


21 


“7 


“ 82.2 


0 


3601 


14 


8 


6 


“ 83.2 


0 


3601 


14 


16 




“ 84.2 


0 


3601 


25 


26 


“1 


“ 83.4 


365 


3601 


3 


11 


“8 


" 84.2 


365 


3601 


n 


0 




“ 83.4 


374 


3601 


o 


5 


“3 


“ 33.2 


374 


3601 


4 


5 


“1 


“ 84.2 


374 


3601 


3 


4 


“1 


“ 83.4 


525 


3601 


9 


7 


o 

Sm 


“ 84.2 


525 


3601 


20 


22 


^2 


“ 83.2 


525 


3601 


10 


11 


“1 


GROUP 


MEAH 


VARIANCE N (OF 


ERROR 


TERM) 


365.0 


“ 3.0 


O 

O 

in 


2.0 






374.0 


“ 1.7 


1.3 


3.0 






525.0 


“.3 


4.3 


3.0 






OVERALL : 


“ 1.5 


o 

o 


o 

CD 






COURSE : 


3602 










“ 82.1 


0 


3602 


7 


11 


“4 


“ 82.3 


0 


3602 


15 


14 


1 


“ 83.3 


0 


3602 


15 


16 


“1 


“ 83.1 


0 


3602 


7 


7 


0 


“ 84.1 


0 


3602 


12 


12 


0 


“ 84.3 


0 


3602 


14 


14 


0 


“ 83.3 


374 


3602 


3 


5 


-n 


“ 84.3 


374 


3602 


2 


4 




“ 83.1 


374 


3602 


1 


1 


0 


“ 84.1 


374 


3602 


3 


3 


0 


“ 84.3 


525 


3602 


12 


10 


2 


“ 83.1 


525 


3602 


6 


6 


0 


“ 83.3 


525 


3602 


11 


11 


0 


1 

CD 


525 


3602 


9 


9 


0 


“ 83.3 


686 


3602 


1 


0 


1 


GROUP 


MEAN 


VARIANCE N (OF 


ERROR 


TERM) 


374.0 


" 1.0 


1.3 


4.0 






525.0 


* 5 


1.0 


4.0 






686.0 


1.0 


1.0 


1.0 






OVERALL : 


“.1 


1.6 


9.0 







Figure A. 5 Output from Input Sequence shown in 
Figure A.4 : COOESE Program- 
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9Coursec03^ 



7 COURSE matrix;xji;y;z;mc;vc;j;x2?Y2»hq;vq;w;qtr;hq;nc;ns;we;xwd;ae; 



C13 R 

C23 R this FUNCTXOM, given ftN INPUT MATRIX IN THE FOULOMING FORM : 

C33 R 

C43 n OTR CURRIC CSE ACTUAL PREDICTED ( ACTUAl PREDICTED) 

C53 R 



C63 


* 


WILL SORT AND DISPLAY 


THE DATA 


BY CURRICULUM 


IN THI 


C73 


ft 


FOLLOWXt-ZG MANNER ; 














C83 


ft 
















C93 


ft 


COURSE 














C103 


ft 


QUARTER 


1 


DATA 




CURRICULUM 


1 


DATA 


C113 


ft 


QUARTER 


2 


DATA 


OR 


CURRICULUM 


2 


DATA 


C123 


ft 


QUARTER 


3 


DATA 




* 






C133 


ft 


QUARTER 


4 


DATA 




CURRICULUM 


N 


DATA 


C143 


ft 


MATRIX OF GROUP 


MEANS , 


VARIANCES AND N 







C153 R OVERALL MEAN, VARIANCE AND N 

C163 R 

C173 R ^ MATRIX OF ACCURACY, PREDICTIVE AND NO ERROR PERCENTAGES IS 

C133 R available if desired, 

C193 R 

C203 R (ALL QUARTERS WILL BE DISPLAYED AS NEGATIVE NUMBERS, THIS ALLOWS 
C213 R CORRECT PLACEMENT OF THE OUTPUT COLUMNS WITHOUT RESORTING 



C223 R COMPLEX CHARACTER ARRAY MANIPULATIONS,) 

C233 R 

C243 R ******** NOTE ; THE MEANS AND VARIANCES COMPUTED APPLY 
C253 R ******** ONLY TO THE XNDIVXDUAl, CURRIgULQ [illi ** 

C263 R ******** any summary data (^column 2 entry = O) ** 

C273 R ******** xs compressed out of the computation, X* 

C283 R ******** THIS CAN BE CHANGED BY REPLACXNG THE ^ X* 

C293 R ******** SIGN IN THE LOOPS (WHERE Z OR W C 9 21 ** 

C303 R ******** COMPRESSING COLUMN ^ OR WHERE X2 XS x* 

C313 R ******** COMPRESSING THE OUTPUT) WITH •=', THISx*** 

C323 R ******** FUNCTION WILL THEN DISPLAY ONLY THE x* 

C333 R ******** SyWiiARY DATA, X* 

C343 R 

C353 R COURSES MAY BE SELECTED INDIVIDUALLY IF SO DESIRED, 

C3A3 R 

C373 MATRIX<.MATRIXX (f matrix ) f "111111 

C383 "fOU WANT all courses displayed, enter 1* 



C393 • OTHERWISE, ENTER Q TO SELECT SOME SUBSET OF THE COURSES* 

C403 -►JUMPixtO 

C413 r'THE FOLLOWING COURSES ARE AVAILABLE FOR SELECTION •,* 

C423 ^CJUE matrixc;333 

C433 r'ENTER THE COURSE NUMBERS YOU WANT, ENTER X TO GET THEM ALL,' 

C443 r' (NOTE : YOU must choose MORE THAN ONE COURSE) ' 

C453 Y4-fX4.Q 
C463 ->uuMP2 

C473 JUMP^ • Y4./»X4.XCiX4-UNIQUE MATRIXC;333 

C483 jump2;x«.i 

C493 r'THE DATA MAT BE DISPLAYED (TO LOOK AT PATTERNS) IN GROUPINGS BY* 
C503 r'EXTHER QUARTER OR CURRIC, FOR QUARTER GROUPINGS, ENTER 1 ; FOR' 

C513 r'CURRIC GROUPINGS, ENTER O' 

C523 

C533 r' ACCURACY error, predictive error, and no error percentages ARE' 
C543 r'^VAILABLE, ENTER Y IF DESIRED, N OTHERWISE' 

C553 XND<-a 
C563 CRSLOOPtr* ' 

C573 -►Ox\x>Y 

C583 (r'COURSE : '),^xci3 

C39 3 Ct-(XCX3=:MATRixc ;33 ) / C13 matrix 






PE 



Figure A* 6 



•COORSE* Program 
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C60: «C<-MEAN(ZC J23?«0)/=C J63 

L611 VC<-VARIA(ZC J23?iO)/2C 

L621' (zc;2:;<50)/2Cr63 

L622 J 4 -I 

L6A2 ->NEXTx\Vpfi 

C65: «tr4-iox (-zc; 13)-L-ZC ? i: 

C663 -►OUT 

C673 next : «tr4-zc j 23 

C683 OUT ; Y24-f X24-K2C^J<24 -unique otr^ 

C693 NS 4 -AE 4 -PE 4 -NQ 4 -MQ 4 -VQ 4 . \ 0 

C703 otrloop : 4Endx ) J > Y2 

C713 w<-( (J<2CU3 = Qtr)/ci3 Z) order 6 

C723 N«4-N«F«E«N(wc;2 3?iO)/wC »63 

C733 v«4.vQ,vaRift(wc;23?^0)/wC?63 

C74 3 NQ<-N«,NE4-f (wc ;23?«0)/wC f 63 

C753 4JUMP3x \ IND= • N ‘ 

C76 3 PE4-PE, (T/(«C ;23?f0)/(wc ; 4 3 = 0) vwc 553=0 )^NC 
C773 AE4-aE, (+/( wc;23?«0)/(wc 543/0) A( v<c;53?J0)awc 5 63?iO)*5-NE 
C78 3 ns<-ns, (4./(wc 5 23/0)/( «C 5 4 3/0)a( wc 5 53/0)awc 5 63=0)^ne 
C793 JUMP3:,^w 
C803 t’ • 

C813 U4-J+1 

C823 -►Qtrloop 
C833 END*l4.I + l 

C843 r' GROUP MEaN variance N (OF error TERM)' 

C853 7 1 t(X2/0)/C13( («}X2fC0.53 m«)»C23 vq),C23 nq 

C863 T' • 

C873 (r'ovERALi-: •), ^ 1 tNC,vc,nc 

C883 t' • 

C893 -►UUMP4X \ IND= ' N • 

C903 r' ERROR PERCENTAGES' 

C913 r' SSQUP pre acc none n' 

C923 7 2 r(J<2/0)/C13( ( (««X2fC0*53 f^E),c23 <^E),c23 ns),c:23 nq 

C933 r' • 

C943 JUMP^tr' ' 

C953 -♦CRSLOOP 
V 



Figure A. 6 (continued) 
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INPUT EROMPT 

COOESE OS 

♦ 

0 

♦ 

3601 3602 

♦ 

1 

♦ 

n 



* see COURSE listing for details 
of the prompts 



Figure A. 7 Input Syntax 



T 



COOESE 



60 



r 



COURSE ; 
" 82.2 


3601 

0 


3601 14 


8 


" 83.2 


0 


3601 14 


16 


" 84.2 


365 


3601 2 


0 


" 84.2 


525 


3601 20 


22 


" 83.2 


374 


3601 4 


5 


" 83.2 


525 


3601 10 


11 


" 84.2 


0 


3601 25 


26 


" 84.2 


374 


3601 3 


4 


" 83.4 


0 


3601 14 


23 


" 83.4 


365 


3601 3 


11 


" 82.4 


0 


3601 14 


21 


" 83.4 


374 


3601 2 


5 


" 83.4 


525 


3601 9 


“7 


GROUP 


MEAN 


VARIANCE N (OF 


ERROR 


2.0 


4 6 


2.3 5.0 




4.0 


“3 4 0 


25.0 3.0 




overall; 


"145 


10,0 8.0 





COURSE ; 
" 82.1 


3602 

0 


3602 7 


11 


" 83.1 


0 


3602 7 


7 


" 83.1 


374 


3602 1 


1 


" 83.1 


525 


3602 6 


6 


" 84 . 1 


0 


3602 12 


12 


" 84.1 


374 


3602 3 


3 


" 84 . 1 


525 


3602 9 


9 


" 33.3 


374 


3602 3 


5 


" 84.3 


374 


3602 2 


4 


" 34.3 


525 


3602 12 


10 


" 82.3 


0 


3602 15 


14 


" 83.3 


0 


3602 15 


16 


" 83.3 


686 


3602 1 


0 


" 33.3 


525 


3602 11 


11 


" 84.3 


0 


3602 14 


14 


GROUP 


MEAN 


VARIANCE N (OF 


ERROR 


1.0 


.0 


.0 4.0 




3.0 • 


"42 


3.2 5.0 




OVERALL : 


"4 1 


1.6 9.0 





TERM ) 



"1 

1 

0 

0 

TERM) 



Figure A. 8 Output from Input Sequence shown in 
Figure A. 7 : COURSE Program* 
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Cur ric ula Related Work 

In looking at the curricula information, we want to 
find out just how good the figure ’twelve' is for the 
expected input from the Antiwhale Warfare curriculum. Since 
curriculum 374 will be administering the program, we will 
use the data from 374 as an approximation of curriculum 
449. 5 's performance. Again, the variable 'OS' contains the 
information we want. The 'CURRICP' program most easily 
shows this data. Through proper input, we can call up the 
historical performance of curriculum 374. The 'CURRICP' 
program provides all the prompts necessary {see figure A. S) . 
Since we are only interested in one curriculum but two are 
required for proper program execution, a dummy curriculum 
(333) is requested. The program output is shown in figure 
A. 10 See figure A. 11 for a program listing. 

As we can see from figure A. 10, curriculum 374 
routinely appears to overestimate the number of students 
expected for any given course. To show more adequately 
patterns of concern in the data (such as performance for 
OS3601 and OS3602) , we can look at the 'CORRIC program 
sorted by course. This will break out the predictions to 
better show how well curriculum 374 performs for individual 
courses. Figure A. 12 shows the inputs, figure A. 13 the 
outputs, and figure A. 14 the program itself. 

As we can now see from the course data shown in 
figure A. 13, curriculum 374 generally overstates the 
expected number of students attending courses OS3601 and, to 
a lesser extent, OS3602. We can't observe any possible 
patterns by quarter due to the limited number of observa- 
tions. A brief look at the error percentage table shows no 
predictive errors for either course (they're taught or not 
taught when they say so - no 'zeros* in the actual or 
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predicted columns), and an improvement in accuracy from 3601 
to 3602 { 0 % error free to 50% error free respectively) . 

In summary, through use of the curricula related 
programs we have found that a slight overestimation of 
students expected is usual for these courses. This trend may 
or may not hold for increased class sizes (of, say, twelve 
students vice three or four) . Therefor, combined with the 
information we previously obtained from the course related 
programs, we conclude that new sections will probably not be 
required. 



INEUT PBOM FI 

CUERICP OS 

♦ 

0 

* 

374 333 

♦ see COURSEP listing for details 
of the prompts 



Figure A. 9 Input Syntax ; CORRICP 
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1 



CURRICULUM ; 374 



"83.4 


374 


2101 


0 


6 


"6 


"83.4 


374 


2103 


T 


6 


"4 


"83.2 


374 


2103 


1 


4 


"3 


"83.3 


374 


3604 


1 


4 


"3 


“83,4 


374 


3601 


n 


5 


"3 


~83,3 


374 


3602 


3 


5 


■*2 


"83.4 


374 


3603 


4 


6 


— 2 


~84. 1 


374 


3604 


2 


4 


“2 


"84.3 


374 


3602 


2 


4 


~n 


"83.2 


374 


3601 


4 


5 


"1 


"83.2 


374 


3604 


0 


1 


"1 


"84.1 


374 


2103 


0 


1 


"1 


"84 . 1 


374 


3401 


0 


1 


"1 


"84.2 


374 


2103 


2 


3 


"1 


"84.2 


374 


3601 


3 


*4 


"1 


"84.3 


374 


3604 


■7 


3 


"1 


"83.1 


374 


3602 


1 


1 


0 


"83.1 


374 


3604 


4 


4 


0 


"84 . 1 


374 


3602 


. 3 


3 


0 


"84.1 


374 


3603 


1 


1 


0 



CURRICULUM ♦ 333 



Figure A. 10 Output froa Input Sequence shown in 
Figure A. 9 : CDBEIC? Program. 



VCURRICPCQ3^ 



C13 

C23 

C33 

LAI 

C53 

162 

172 

LQ2 

192 

C103 

C113 

L122 

C133 

Z1A2 

L152 

1162 

Z172 

Z1B2 

L192 

Z202 

C213 

C223 

C233 

C243 

C253 

C263 

Z272 

C283 

Z292 

Z202 

C313 

C323 

C333 

C343 

C353 

C363 

C373 

C383 

C393 

C403 

C413 



^ CURRICP MATRIX I ; Y ; z 

ft THIS FUNCTION, GIVEN AN INPUT MATRIX IN THE FOLLOWING FORM I 
ft 

ft QTR CURRIC CSE ACTUAL PREDICTED (ACTUAI PREDICTED) 

A 

ft WILL SORT AND DISPLAY THE DATA BY CURRICULUM IN THE 
ft FOLLOWING MANNER t 
A 

ft CURRICULUM 

A 

ft DATA ORDERED BY MAGNITUDE OF ERROR TERM 

A 

ft (ALL QUARTERS WILL BE DISPLAYED AS NEGATIVE NUMBERS* THIS ALLOWS 
ft FOR CORRECT PLACEMENT OF THE OUTPUT COLUMNS WITHOUT RESORTING 
ft TO COMPLEX character ARRAY MANIPULATIONS*) 

A 

ft THE GOAL OF THIS PROGRAM IS TO DISPLAY OBVIOUS PATTERNS THAT 
ft MAY THEN BE LOOKED AT MORE CLOSELY USING THE 'COURSE* OR *CURRIC* 
ft FUNCTIONS* CURRICULA MAY BE SELECTED INDIVIDUALLY IF SO DESIRED* 

A 

MATRIX4-MATRIXX ( f MATRIX ) f ”111111 

^*IF YOU want all CURRICULA DISPLAYED, ENTER 1' 

^'OTHERWISE, ENTER Q TO SELECT SOME SUBSET OF THE CURRICULA* 
-►JUMPlx \D 

' THE FOLLOWING CURRICULA ARE AVAILABLE FOR SELECTION t* 

^X4-XCiX4-UNIQUE MATRIXQJ233 

* ENTER THE CURRIC NUMBERS YOU WANT* ENTER X TO GET THEM ALL** 
^*(NOTE : YOU MUST ENTER MORE THAN ONE CURRICULUM)* 

T4-PX4-Q 

->JUMP2 

J UM PI JY«-fX4-XC^X ^UNIQUE MATRIXQ J 23 3 
JUMP2 J 14-1 
CRCLOOPt^* * 

40X \ I > T 

( r ' CURRICULUM : *),-rXCI3 

z«.(Xci3=>^<»trixc;23)/C13 matrix 

ORDER 6 
I4-I + 1 
r ' ' 

T ' ' 

-»CRCLOOP 



Figure A. 11 'COREICP* Program 
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1 



INPUT FHOHP T 

CUERIC OS 

♦ 

0 

♦ 

S'Jil 333 

♦ 

3 

♦ 

y 



* see COURSE listing for details 
of the prompts 



Figure A- 12 Input Syntax ; CORRIC 
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CURRICULUM ; 374 

^33.4 374 



• 33.4 
"83.2 
“34. 1 

•34.2 

• 34 . 1 

•83.4 

•83.2 

• 34.2 

•33.3 

•34.3 

•83.1 

• 34.1 

• 33.4 

“34.1 

•33.3 

"84.1 

"83^2 

"34.3 

"83.1 



374 

374 

374 

374 

374 

374 

374 

374 

374 

374 

374 

374 

374 

374 

374 

374 

374 

374 

374 



2101 

2103 

2103 

2103 

2103 

3401 

3601 

3601 

3601 

3602 
3602 
3602 

3602 

3603 

3603 

3604 
3604 
3604 
3604 
3604 



6 

4 
1 

3 

1 

5 
5 

4 

5 
4 
1 

3 

6 
1 

4 
4 
1 

3 

4 



CROUP 


MEAN 


variance n 


(OF E 


2101.0 


•6.0 


1.0 


1 .0 




2103.0 


•2.3 


2.3 


4.0 




3401.0 


•1.0 


1.0 


1 .0 




3601.0 


•1.7 


1.3 


3.0 




3602.0 


•1.0 


1.3 


4.0 




3603.0 


”1.0 


2.0 


2.0 




3604.0 


”1 .4 


1.3 


5.0 




OVERALL 


: “1.7 


2.3 


0 

0 

n 




ERROR 


PERCENTAGES 






GROUP 


PRE 


3C£ 


NONE 


N 


2101.00 


1.00 


.00 


.00 


1 .00 


2103.00 


.25 


.75 


.'00 


4.00 


3401 .00 


1.00 


.00 


.00 


1.00 


3601.00 


.00 


1.00 


.00 


3.00 


3602.00 


.00 


.50 


.50 


4.00 


3603.00 


.00 


.50 


.50 


2.00 


3604.00 


.20 


.60 


.20 


5.00 



CURRICULUM ; 333 

CROUP MEAN VARIANCE N (OF ERROR TERM) 



OVERALL : 1^0 



.0 



.0 



ERROR PERCENTAGES 

559y2 RPg ACC NONE 



Figure A. 13 Output from Input Sequence shown in 
Figure A. 12 : CORBIC Program. 
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yCURRICCaiV 



C13 

C2D 

C33 

C43 

C53 

161 

Z71 

C83 

C93 

CIOD 

C113 

C123 

C13J 

C143 

C153 

C163 

C17D 

C183 

C193 

C203 

C213 

C223 

C23J 

C243 

Z252 



T 7 cuRRic MATRix;;<;ifYjz;Mc;vcjj;x2;*»'2JMajvQ;w;A;Ns;NE;iHD;AE;pE;v;Nc;Ha 

A 

n THIS FUNCTION, GIVEN AN INPUT MATRIX IN THE FOLLOWING FORM • 

A 

(I GTR CURRIC CSE ACTUAL PREDICTED ( A C T U AL -F R E D I C T E D ) 

A 

n WILL SORT AND DISPLAY THE DATA BY CURRICULUM IN THE 
n FOLLOWING MANNER * 

A 

n CURRICULUM 

A QUARTER 1 DATA COURSE 1 DATA 

A QUARTER 2 ^lATA OR COURSE 2 DATA 

A QUARTER 3 DATA , 

A QUARTER4DATA COURSE N DATA 

A MATRIX OF GROUP MEANS, VARIANCES AND N 

A OVERALL MEAN, VARIANCE AND N 

fl 

A A MATRIX OF ACCURACY, PREDICTIVE AND NO ERROR PERCENTAGES IS 
A AVAILABLE IF DESIRED* 
fl 

A (ALL QUARTERS WILL PE DISPLAYED AS NEGATIVE NUMBERS, THIS ALLOWS 
A FOR CORRECT PLACEMENT OF THE OUTPUT COLUMNS WITHOUT RESORTING 
A TO COMPLEX CHARACTER ARRAY M AN I PUL AT I ONS , ) 
fl 

A CURRICULA MAY BE SELECTED INDIVIDUALLY IF SO DESIRED, 

A 



C263 MATRIXl-MATRIXx ( f MATRIX ) f ”111111 

Z272 '''Ou want all curricula displayed, enter i* 

C283 t* ' otherwise , enter O to select some subset of the curricula' 

C293 4UUMPix\0 

C30D t'THE following curricula are available for selection •' 

C313 i»X4-xc4X«-unique matrixc;233 

C323 t'EHTER THE CURRIC NUMBERS YOU WANT, ENTER X TO GET THEM ALL,' 

C333 t*'(^*OTE • YOU MUST ENTER MORE THAN ONE CURRICULUM)' 

C343 ri-fX^-Q 
C353 ->JUMP2 

C363 JUMPi ; Y«.px<-xc4kJ<<-UNiQUE matrixc;233 

£272 JUMP2:i<-l 

C383 t'THE DATA MAY BE DISPLAYED (TO LOOK AT PATTERNS) IN GROUPINGS BY* 
C39D t'EITHER QUARTER OR COURSE, FOR QUARTER GROUPINGS, ENTER 1 J FOR' 
C403 t'OOURSE GROUPINGS, ENTER 3,' 

C4n 

C42D t'^OCURACY ERROR, PREDICTIVE ERROR AND NO ERROR PERCENTAGE MATRIX?* 

C433 t ' enter r or n* 

C44D INDtQ 




Figure A. 14 



’CORSICA Program 

68 



1602 w<-( (>J2C J3 = A)/C1] 2) ORDER s 

1612 Ma«-Ma,MEAH wtJ 6 ] 

1622 va<-va,vARiA ^Z*,62 
C63] Ha4.NQ,HE<-pwc 56] 

L6A2 -»JUMP3x \ IND= • M • 

C65] PE<-RE, ( +/( «c 5 4 J = 0 ) vWC ;53 = o )^ME 
1662 «e<-ae, (+/<wc f4]?£0) a(WC 

C67] ns<-h 5, (+/(wc; 43 ,iO)A(wc ; 5 ] ^0 ) a wq ; ^ 3 = 0 ) *^WE 

Z6B2 JUMP 3 :^w 
C 6 ?: r' • 

L702 + l 

C713 -»C»TRLOOP 
C72J ehd:i<-i + i 

Q73^ GROUP MEAN VARIANCE N (OF ERROR TERM)' 

L7A2 7 1 t( (8iJ<2>C0*5] mo),c23 vo),C23 wq 
C 75:i r' • 

L762 (r'OVERALL : '), 6 1 r»^c,vc,HC 

C773 r* ' 

6783 4 JUMP 4 X ) ind= ' H • 

C7?3 t' error PERCENTAGES' 

C803 r' 5BSyE 8 £C hone N' 

C813 7 2 r( ( (l«J<2f C0.53 fL22 AE)fC23 MS),C23 hq 

C823 1 -' • 

C833 JUMP4:r* ' 

C843 4 CRCL 00 P 
7 



Figure A. 14 (continued) 



D. DATA COLLECTION AND MAINTENANCE 

To provide the matrix of historical data to be displayed 
and analyzed, two sources are used. The prediction data 
comes from the course loading forecast delivered to the 
department chairmen (see figure A. 15). The easiest way to 
compile that data is to obtain a copy of the individual 
course predictions and, on the forecast itself, change the 
letter codes found in the "S/D/T" column into curricula 
numbers (using the coding list contained in appendix B) with 
associated totals. For example, the totals you would write 
in from figure A. 15 for OS3602 are 525; 9, 374: 3, and 0 

(total) ; 12. The ether document required is the course 
grade sheet held by the registrar (see figure A. 16). This 
information is subject to the privacy act and requires a 
written request for permission from the department chairman 
involved to the office of the registrar for access tc the 
files. The records themselves are kept according to 

academic year. This notation is confusing; to prevent prob- 
lems, ask for something like the *83-84 school year* data. 
These records are not allowed to leave the office, and at 
last check were not available on the computer. The data 
must he recorded by hand. A convenient format is: 

Quarter Course Section Total# #Attending/C urricula 

Once the above listed data have been collected and are 
in front of you and a computer terminal, a CMS file as shown 
in figure A. 17 can be created by combining the information. 
The columns correspond to the quarter, curriculum, course, 
actual and predicted number of students. The CMS file thus 
created can be transferred to APL and named, for instance, 
eSNEXT through the following command sequence: 

(get into APL) 

) LOAD 990 CMSIO 
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) ;»'SID HOLD 
OSNSXT <= CHSREAD 

(follow prompts to read your CMS file) 

) ERASE CMSIO 
) SAVE 

NOTES : 1) The API variable ’OSNEXT' now exists 

on your disk in the APL workspace ’HOLD* 

2) The *<=' character shown corresponds to 
the left-pointing assignment arrow in APL 

To add the (Actual minus Predicted) term to the matrix, 
enter : 

CSNEXT <= OSNEXT,[2] 0SNEXT£;4] - OSNEXT[;5] 

To add this matrix to the variable OS (assuming that OS is 
in the workspace), enter: 

CS <= OS,[ 1 ] OSNEXT 

The entire updating procedure should take about one full 
workday for each year of data. 
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Figure A. 15 Individual Course Prediction Format 



1 



OFFICIAL GRADE ROSTER PAGE 1 

NAVAI POSTGRADUATE SCHOOL 
MONTEREY, CALIFORNIA 93943 DATE 13 APR 

COURSE NO. OS3602 SEGEMENT 00 QUARTER 1 AY 83-84 

COMBAT MODELS PROF. SHUDDE, REX H. 



3 LECTURE HCURS 1 LABORATORY HOURS 





NAME 






RANK 


CORPS/ 

COUNTRY 


CURR 


MARK 


1. 


xxxxxx^ 


xxxx 


X 


LT 


DSN 


#374 


A 


2. 


xxxxxx^ 


xxxx 


X 


LCDR 


USN 


#374 


B 


3. 


xxxxxx^ 


xxxx 


X 


LCDR 


USN 


#525 


B 


4. 


xxxxxx. 


xxxx 


X 


LT 


USN 


#525 


B 


5. 


XXXXXXr 


xxxx 


X 


LCDR 


USN 


#525 


Pi- 


6. 


xxxxxx. 


xxxx 


X 


LT 


USN 


#525 


Pi 


7. 


xxxxxx^ 


xxxx 


X 


LCDR 


USN 


#374 


A- 


8. 


xxxxxx. 


xxxx 


X 


LT 


USN 


#525 


A 


9. 


xxxxxx. 


xxxx 


X 


LT 


USN 


#525 


B 


10. 


xxxxxx. 


xxxx 


X 


LT 


USN 


#525 


B 


1 1. 


xxxxxx. 


xxxx 


X 


LCDR 


DSN 


#525 


B 


12. 

FCEM 


xxxxxx, 

4 


xxxx 


X 


LT 


USN 


#525 


B 


THIS 


IS THE 


OFFICIAL 


GRADE 


ROSTER TO 


BE FILED I 



ACACEHIC DEPARTMENT 



Figure A. 1 6 



Sample Grade Sheet (names removed) 
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84.1 


0 


2103 


49 


44 


84.1 


365 


2103 


29 


28 


84 . 1 


366 


2103 


1 1 


15 


84.1 


374 


2103 


0 


1 


84. 1 


590 


2103 


4 


0 


84 . 1 


591 


2103 


2 


0 


84 . 1 


535 


2103 


1 


c 


84.1 


600 


2103 


2 


0 


84.1 


0 


3001 


25 


22 


84.1 


368 


3001 


25 


22 


84.1 


0 


3004 


28 


28 


84.1 


367 


3004 


28 


28 


84.1 


0 


3006 


63 


59 


84.1 


837 


3006 


21 


22 


84.1 


857 


3006 


1 1 


10 


84.1 


81 9 


3006 


2 


2 


84.1 


827 


3006 


2 


2 


84.1 


£47 


3006 


13 


13 


84.1 


815 


3006 


5 


4 


84.1 


813 


3006 


1 


1 


84.1 


817 


30 06 


5 


1 


84.1 


36 0 


3006 


2 


0 


84.1 


620 


3006 


1 


0 


84.1 


0 


3102 


56 


68 


84.1 


620 


3102 


2 


2 


84,1 


837 


3102 


22 


23 


84.1 


£57 


3102 


8 


10 


84.1 


819 


3102 


0 


4 


84 . 1 


£27 


3102 


2 


3 


84.1 


815 


3102 


6 


7 


84.1 


817 


3102 


8 


2 


84 . 1 


814 


3102 


6 


6 


84 - 1 


813 


3102 


1 


•3 


84.1 


847 


3102 


1 


6 


84.1 


0 


31 04 


29 


10 


84.1 


817 


3104 


4 


1 


84.1 


570 


3104 


10 


7 


84 . 1 


36 0 


3104 


1 


2 


84.1 


813 


3104 


2 


0 


84.1 


819 


3104 


4 


0 



Figure A. 17 CMS File Format 
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E. 01HEE AVAILABLE PEOGRAMS AND DATA 



A ccmplete listing of the functions available in the 
•OSDATA’ workspace is shown in table VII - The COURSEP, 
COURSE, COREICP and CURRIC programs have been previously 
demonstrated. The oily non-trivial function remaining to be 
shown is the 'WILCOXSE’ program (see figure A. 18) . As shown 
in the figure, this program is based on the Wilcoxian Signed 
Rank Test»* detailed in [Ref. 5 pp 280-288]. 

A self-descriptive listing of the variables contained in 
the workspace is shown in table VIII . The latest data pres- 
ently available is the spring of 1984 (academic year 83-84) 
figures. 



1 »For a complete description of the various hypotheses 
that may be explored, see [Ref- 5 pp 280-288]. In the body 
of the thesis, the magnitudes of the forecasting . errors (y) 
were tested against the magnitudes of prediction errors 
derived from course matrices (x) . The hypothesis tested 
(with rejection the desired result) was that the *x' values 
tend to be equal to or larger than the *y' values. This was 
one of six possible interpretations of the null hypothesis. 
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TABLE 7II 

Functions Available 



Function 


Syntax 

♦ •(; 3tc 4c 4c :4e:(c 4c 


Description 


CCUESEF 


COURSEP mtx 


(previously shown) 


CCOESE 


COURSE mtx 


(previously shown) 


CORRICP 


CURRICP mtx 


(previously shown) 


COERIC 


CURRIC mtx 


(previously shown) 


WIICOXSR 


y BILCOXSR x 


Shows the Wilcoxian Signed 
Rank Test results (see 
description in text) 


ORDER 


A ORDER E 


Orders rows of matrix A 
according to decreasing mag- 
nitude of column B, returning 
the ordered matrix 


UNIQUE 


UNIQUE & 


Returns the unique elements 
of the vector or matrix A in 
ascending order 


MEAN 


MEAN X 


Returns the arithmetic mean 
of vector X 


VARIA 


VARIA X 


Returns the unbiased estimate 
of the variance of vector X 
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r 



1 



^WILCOXSR CD39 
^ Y wiLCOKSR ;<;d;t;r 

C13 ft 

C23 ft this function provides the wilcoxian sighed rank test 

C3.3 ft STATISTIC (FROM CONOVER PG , 280 )* INPUTS X AND Y ARE 

C4] R DATA VECTORS OF EQUAL UgNGTH , 

l 53 ft 

C63 D<-(D;dO)/D<-T-X 

C73 ( rank I D ) XD^ 1 D 

C31 T^-(^./R)^(^./R*2) to *5 

C93 (r'TEST STATISTIC s ' ) f T T 

V 



Figure 1.18 



Bilcoxiau Signed Reuik Test Program 
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TABLE VIII 
Variables Available 


VAEIAEIE 


SHAPE 


BESCEIPTION 




:$c :$c :$c :4c :$c 




OS82 


6 9x6 


AI (Academic yearj 81-82 data; 
includes summary xnformation only 
(no data for individual curricula) 


0S83 


255x6 


AY 82-83 data; complete listing 


OSIEK83 


140x8 


AY 82-83 non-entry course data. The 
additional columns contained in this 
matrix (columns 6 and 8) correspond 
tc an 'adjusted prediction' and an 
'adjusted error' term, derived from 
the course matrices. This array was 
used to check the feasability of 
making adjustments based solely on 
the 'track' a student takes using the 
Wilcoxian Signed Bank Test previously 
mentioned. 


OS84 


22 8x6 


AY 83-84 data; only gtrs 1,2 and 3 


CS 


552x6 


Combination of all of above 
(without repetition) 


OA83 


113x9 


This matrix is set up in a different 
format from those previously dis- 
cussed. The nine columns represent 
Quarter, OA course number. Total 
attending, USN attending, USA 

attending, USMC attending. Others 
attending. Total predicted, and Total 
attending minus total predicted, 
respectively. The original intent, 
dropped due to time constraints, was 
tc use these figures to aid in OA 
elective scheduling by looking at 
expected number of upcoming free 
electives by service through use of 
the OA course matrix. 

. 
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APPENDIX B 
SCHEDOLING CODES 



This appendix contains an alphabetical listing of the 
specialty codes used in the scheduling process as of August 
1984. This listing only uses the first two letters of any 
individual coding because the remaining letters specify 
either the degree awarded or the degree and month of input, 
neither cf which charge the curriculum associated. Using 
these codes, numbers of students predicted by any given 
curriculum for any given course can be taken from the indi- 
vidual course forecast promulgated by the programming 
office. Code 0301 at extension 2773 at the Naval 

Postgraduate School can be contacted for updated 
information. 

Specialt y 
Curricula Degree 
Numbers Codes Program 

:4c :4c :9c :4c :4c :4c :4c 4c :4c :4c :4c 4c4c4c3fr:4c4‘4c4c4c4c4c4c :4c 4c 4c 4c 4c 4c # 4c 4c 4c 4c 4c 4c 4c 4c 4c 4c 4c 4c 4c 4c 4« 4c 4c 



6 1 0 AC — 

AT — 

6 1 1 AX — 

368 CS — 

600 DC — 

590 EC — 

ED — 
EG — 
ER — 

38 0 GH — 



Aero Engr, incl Engr Sci 



Aeroelectronics 

Computor Science 

Electrical Engineering (Comms) 
Note: DCTR is curriculum 460 

Electrical Engineering ; various 
sub-specialties. Note: EETR is 
curriculum 460 



Math option of advanced science 
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Specialt y 
Curricula Degree 
Numbers Codes Program 



381 


GP — 


Physics option of adv. science 


620 


HC — 


Communications Management 




HH — 






525 


IX — 


Antisubmarine Warfare Note: lETR 






is curriculum 460 


365 


JM — 


Command, Control and Communica- 




JN — 


tions ; various sub-specialties 




JO — 


Note: J CT E 


and JXTR are from 




JP — 


curricula 


460 




JS — 






595 


KE — 


Electronic 


Warfare Systems Tech- 






nology Note: KETR is curric 460 


817 


ME — 


Admin Sci 


- Econ, OSMC 


837 


ME — 


Admin Sci 


- Financial mgt 


857 


MH — 


Admin Sci 


- Human Resource mgt 


819 


MI — 


Admin Sci 


- Inventory 


817 


MK — 


Admin Sci 


- International stu. 


827 


MM — 


Admin Sci 


- Material 


817 


MO — 


Admin Sci 


- Ops Research, Army 


847 


MP — 


Admin Sci 


- Personnel 


815 


MR — 


Admin Sci 


- Procurement 


817 


MS — 


Admin Sci 


- MGMT Sci, Coast Guard 


8 14 


MT — 


Admin Sci 


- Transportation 
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Specialty 
Curricula Degree 
Numbers Codes Program 



8 13 


MV — 


Admin Sci - Material Movement 


817 


MX — 


Admin Sci - Civ. Federal Program 


570 


NE — 


Mechanical Engineering 




NF — 


ME via 460 




NI — 


ME via 460 




NE — 


ME via 460 




NT — 


ME via 460 


44 1 


OH — 


Hydrographic Science 


440 


OS — 


Oceanography Note: OCTR is 460 


367 


PL — 


Information Systems (from CS ) 


360 


RA — 


Cps Analysis, Army • 


460 


EC — 


Zngr Sci for Ops Analysis 


360 


EG — 


Cps Analysis, General 




EN — 


Navy 




RS — 


Supply 




RP — 


PHD program 


591 


SE — 


Space Systems Engineering 


366 


SO — 


Space Systems Operations Note; 
SXTR and SOTR are curriculum 460 


535 


UX — 


Dnderwater Acoustics Note: UETR 



is curriculum 460 
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Specialty 



Curricula 


Degree 




Numbers 


Codes 


Progra m 






530 


WA — 


Weapons Systems Technology 




nc — 






WE — 






WH — 




532 


WN — 


Weapons Systems Science 


530 


WP — 


Weapons Systems Technology 


531 


WS — 


Weapons Systems Science 


372 


XA — 


Meteorology 


373 


XS — 


Met. and Oceanography 


37U 


XT — 


Air-Ocean Tactical Env. Support 






Note: XOTE'and XXTR are 460 


683 


YA — 


NSA - OSSR, Europe; USN 


682 


YB — 


NSA - Asia/Pacific; USN 


681 


YC — 


NSA - Middle East, Africa; USN 




YD — 


- S. Asia; USAF, USA 


682 


YE — 


NSA - Asia/Pacific; USAF, USA 


681 


YF — 


NSA - S. Africa; USAF, USA 




YG — 


- N. Africa; USAF, USA 




YM — 


- Middle East; USAF, USA 


687 


YN — 


NSA - Nuclr/Plng; USN 


684 


YO — 


NSA - Intl/Org/Neg; USN 
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Specialty 
Curricula Degree 
Numbers Codes Program 

:{c 4 c :}c :(c 4t 4t 4t 4c 4^ ^ 4 ^ ❖ ^{c 4 t 4t 4t 4c 4t 4c 4c 4c 4c 4c * :4c 4c 4^ ❖ ❖ ^ 4c 4c 4= 5^c 4c 4c 4c 4c ❖ 3|s 4c 4c 4c 4c 4c 



686 


YP — 


NS A 


- Strat/Plng; USN 


683 


YE — 


NSA 


- DSSR/E. 


Europe 


825 


YS — 


NS A 


- Navy Intell; DSN 


683 


Y¥ — 


NSA 


- OSSR/W. 


Europe 
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APPENDIX C 

DETAILED SOMMABY OF RESULTS 

This appendix contains a brief descripti 
results found for each course and curriculum. Th 
are listed in ascending numerical order firs 
courses then the curricula. Courses or curricula 
problems are flagged with a ’P* in front of the 
number. 



Course 

Number Comments 

P 2101 Not taught, only predicted. 

P 2102 Large magnitude errors, generally undere 
Curriculum 590 causes most of the proble 
ulum 600 to a lesser extent. No guarterl 

P 2103 Generally good except for curriculum 531 
never sends people but has predicted 40, 
No significant quarterly pattern. 

P 3001 Prediction errors in third and fourth gu 
not predicted but taught. Only curriculu 



3002 


Generally 


gocd. Only 


taken 


by c 




others ar< 


e predicted. 


Need 


more 


3003 


Generally 


good. Need 


more 


data. 


3004 


Gener ally 


good. 






3005 


Generally 


good. Need 


more 


data. 



on of the 
ese results 
t for the 
that cause 
ident ifying 






stimates . 
ms, curric- 
y pattern. 

which 
27 and 7. 

arters - 
m 368 . 

825 t ho ug h 
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Course 



Number 


Comments 


d^d^ 


d^i^d^d^di: d^d^d^d^d^di^ d^d^d^d^d^d^d^d^d^cd^d^d^d^:^:^d^d^d^:^d^:^:^d^d^:^:^:^ d^:^ 3}c3$c:ic:}t:{c:jc:jc:Jc:^c:Jc:ic 


3006 


Generally good except for curriculum 817. Only 
taught first and third quarters. Overall class 
sizes fairly stable at 63 and 87 respectively 
(three sections required) . 


P 3008 


Definite prediction problem that appears to be 
improving. Curriculum 365 provides most of the 
input. 


P 3101 


Curriculum 460 not predicted generally causing a 
significant underestimation (usually results in 
adding an additional section) . More data needed. 


P 3102 


Overall significant overestimation from cumulative 
effect of 8XX curricula. Worst in third quarter. 


P 3103 


Overall significant overestimation from cumulative 
effect of 8XX curricula. No quarterly trend. 


P 3104 


Large errors in summary predictions generally due 
to prediction vice accuracy errors. All curricula 
except 570 and 817 experience these errors. First 
quarter predictions look good, underestimates occur 
in the second quarter, overestimates in the third. 


P 3105 


Significant prediction problems. Generally offered 
only third quarter. 


3302 


Not enough data. 


3303 


Generally good. 525 provides majority of input. 


3401 


Not enough data. 


3402 


Prediction problem, not severe. 
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Course 



Number 


Comments 






3403 


Generally excellent, not enough data. 


P 3404 


Curricula 365 and 825 provide most of the inputs 
to this course. 365 does an excellent job, 825 a 
poor one. 


3601 


Generally good. 


3602 


Generally excellent. 


3603 


Generally excellent. 


3604 


Generally good. 


P 3702 


Significant prediction problem. Not taught first 
quarter, only predicted. Curriculum 847 provides 
most of the input. 


4601 


Generally excellent. 


4602 


Generally excellent, needs more data. 


4701 


Generally gocd. 



:4c :fc :4c :jc :4c :(c :}c :4c :4c :4c :$c :4c :4c ^ ^ ^4c :4c :4c :4c :4c :4c :4c :4c :4c :4c :4c :4c :4c :4c :4c :4c :4c :$c :4c :4c :4c :4c :4c :4c :4c :4c^4s ^ :4« :4«34e :4c ^ 34c:4c :4c 

** END OF COORSE LISTING 

♦ :(c4: :«c :(c :(c ^ :{c :{c4: ** * :«! 
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Curriculum 



Numter 


Comments 






P 0 


Summary predictions are innaccurate due to course 
effects. So quarterly trend seen. 


p 360 


Seldom predicted for OS courses. Minor effect due 
to small number of inputs. 


365 


Generally excellent, possible problem with OS3008 


366 


Generally excellent. 


367 


Generally good, need more data in OS31 01- 


P 368 


Third quarter prediction problem developing. 


372 


Not enough data. 


373 


Not enough data. 


P 374 


Tends to overestimate. 


380 


Not enough data. 


440 


Not enough data. 


441 


Not enough data. 


P 460 


Never predicted. 


525 


Generally excellent. 


P 530 


Problem in OS3104. 


P 531 


Significant problems affecting OS2103 and OS3104. 


532 


Not enough data. 


535 


Not enough data. 


570 


Possible errors in third quarter, need more data. 
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Curriculum 

Numter Comments 



:^c :^c :#c :«c :«c 




P 590 


Problem in OS2102 


591 


Looks good, not enough data. 


600 


Generally good. 


610 


Not enough data. 


611 


Not enough data. 


620 


Generally good. 


683 


Mot enough data. 


686 


' Not enough data. 


687 


Not enough data. 


8 13 


Generally excellent. 


8 ia 


Generally excellent. 


815 


Generally fair. 


P 817 


Problem with OS3006, good otherwise. 


P 819 


Various prediction problems. 


P 825 


Problem with OS3404. 


P 827 


Overestimates for OS3102 and OS3103. 


P 837 


Problem in OS3102, prediction error for OS3105. 


P 847 


Problem with OS3702. Good with OS3103. Third 
quarter predictions are poor. 


P 857 


Routine innaccuracies with no apparent pattern. 


999 


Insignificant input. 
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APPENDIX D 



CONDENSED COOBSE MATRICES 

This appendix contains the condensed course matrices for 
all curricula that take OS courses. Curricula 610, 611, 681, 
682, 683, and 684 dc not have any OS courses required and 
are not included, ’xxxxxx* entries in the condensed matrices 
indicate that the students are onboard but not taking OS or 
OA designated courses (these entries are not made following 
the final OS or OA course taken to prevent display clutter). 
The entries are based on the printed course matrices as 
published as of August, 1984. These matrices were provided 
by the individual curricular offices. 
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368 053001 f,sp 

372 xxxxxx xxxxxx CS3104 f,w,sp,s 

373 xxxxxx OS3104 f,w,sp,s 



Condensed Course Matrices (continued) 
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Condensed Course Matrices (continued) 
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appendix e 

ENTRY / TRACK COORSE DESIGNATIONS 

This appendix contains a complete listing of the OS 
courses divided into ’Entry' and ’Follow-on’ catagories. An 
’Entry* course is the first OS course that is taken in a 
given course matrix (for a given curriculum) ; the first time 
a curriculum can be ’seen’ by the Operations Analysis 
Department. ’Follow-cn’ courses are those courses that are 
taken in quarters following the quarter the ’Entry’ course 
is taken. All curricula take one of the six ’Entry' courses 
prior to any further OS course work with the exceptions of 
curricula 610, 611, 681, 682, 683 and 684 which have no 

required OS courses. 



ENTRY 

COURSE 


CURRICULUM 


TRACK (Follow-on) 
COURSES, ALL OS 


OS2 103 


365 


3604,3404,3008, 3603, 4602 




366 


3604,3404,3008,3603,4602 




374 


3601,3602,3603,3604 




525 


3303,3604,3601 ,3401,4601,3602 




535 


none 




595 


3604,3003,3603,3403,4601 


CS3101 


3 67 


3004 




620 


3005 




686 


none 
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ENTRY TRACK (Follow-on) 

COURSE CURRICULUM COURSES, ALL OS 



CS310 1 


687 


rone 




825 


3404,3002 


OS3001 


368 


none 


OS3 104 


372 


none 




373 


none 




440 


none 




441 


none 




530 


none 




531 


none 




532 


none 




570 


none 


CS2102 


590 


3104 




591 


none 




600 


none 


CS3 102 


813 


3103,3006 




814 


3103,3006 




815 


3103,3006 




817 


3103,3006,3604 
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ENTRY 

COURSE 


CURRICULUM 


TRACK (Follow-on) 
COURSES, ALL OS 


OS3 102 


819 


3103,3006 




827 


3103,3006 




837 


3103,3006 




847 


3103,3006,3702 




857 


3103,3006 
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Nava l Postgra d uate Schoo l Cata_^ 3 , 1 983- 1 98 4 
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